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Using Learning Analytics to Assess Student Learning in Online Courses 


Abstract 

Learning analytics can be used to enhance student engagement and performance in online courses. Using 
learning analytics, instructors can collect and analyze data about students and improve the design and delivery 
of instruction to make it more meaningful for them. In this paper, the authors review different categories of 
online assessments and identify data sets that can be collected and analyzed for each of them. Two different 
data analytics and visualization tools were used: Tableau for quantitative data and Many Eyes for qualitative 
data. This paper has implications for instructors, instructional designers, administrators, and educational 
researchers who use online assessments. 
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Introduction 

The landscape of technology in education has changed over the last two decades. Online learning 
has become prevalent. In Fall 2012, there were 6.7 million students taking online courses in higher 
education (Allen and Seaman, 2013) and 1.8 million in K-12 setting (iNacol, 2012) in the United 
States. Constant monitoring and analysis of information through learner-centered instruction and 
assessment are two essential conditions required for the success of today’s online courses. In this 
study, we examine online learner-centered assessment and how it helps with online teaching and 
learning to measure the students’ progress, and take corrective measures if necessary, through the 
lens of learning analytics. Learning analytics focuses on the transformation of education, by 
changing the very nature of teaching, learning, and assessment (Siemens and Long, 2011). 
Learning analytics is defined by the Society for Learning Analytics Research (SOLAR) as “the 
measurement, collection, analysis and reporting of data about learners and their contexts, for 
purposes of understanding and optimizing learning and the environments in which it occurs” 
(SOLAR, p.l). In the next section, we review different types of assessments in online learning and 
the use of learning analytics in assessments. The term assessment is used to capture both formative 
assessment (activity) and summative assessments. 

Learner-centered assessments shift the move from grades, marks and credits to learning, 
outcomes, and graduating with the skills needed as a professional. Researchers have studied 
assessment in online learning for a number of years (Rovai, 2000; Kim, Smith, and Maeng, 2008). 
In the previous years, practitioners and researchers were primarily using tests, projects as 
assessments, and rubrics to grade students’ performance. Recently, researchers have begun 
promoting and advocating the use of learning analytics which is “interpretation of a wide range of 
data produced by and gathered on behalf of students in order to assess academic progress, predict 
future performance, and spot potential issues” (Johnson et al., 2011, p.28). Macfadyen and 
Dawson (2010) mined data from the Learning Management System (LMS) and studied the 
relationship between student LMS use (e.g., posting discussion messages, completing quizzes) and 
academic achievement. They also stated that “pedagogically meaningful information can be 
extracted from LMS-generated student tracking” (p.l). Fritz (2011) used the “check-my-activity” 
tool to study the relationship between student performance and activity in the LMS. They found 
that students earning a D or F used the LMS 39% less than students earning a grade of C or higher. 
Arnold and Pistilli (2012) used an application called signals which was developed to provide 
instructors the opportunity to use the power of learner analytics to intervene and provide feedback 
to students who were not doing well in their courses. In the next section, we review different types 
of assessments in online learning, and differentiate formative and summative assessments. 

Learning Analytics and Assessment 

Gordon Commission (2013) recommends “separate responsibility for the use of data drawn from 
rich descriptions of these transactions for administrative and for student development purposes. 
Teachers would be enabled to interpret these data diagnostically and prescriptively” (p.l5). 

Gordon commission went further to differentiate between assessment “of’ educational outcomes 
and assessment “for” teaching and learning. Assessment of educational outcomes, is described as 
the “use of assessment for accountability and evaluation” (p.xvi), and assessment for teaching and 
learning is described in “its use for diagnosis and intervention” (p. 163) thus focusing more on its 
formative functions and nature. McManus (2008) from North Department of Public Instruction 
defined Formative assessment as “a process used by teachers and students during instruction that 
provides feedback to adjust ongoing teaching and learning to improve students’ achievement of 
intended instructional outcomes” (McManus, 2008, p.3). As opposed to summative assessment. 
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this definition of formative assessment emphasizes the terms as a process during ongoing 
instruction, where both teachers and students use evidence of current learning to improve 
subsequent learning. 

Summative assessment on the other hand focuses on assessing learning at the end of instructional 
unit, and comparing and benchmarking it with predefined standards. The main purpose of 
summative assessment is to evaluate learners’ achievements with respect to previously defined 
expected competencies. Therefore summative assessment is used more from an evaluative purpose 
rather than a diagnostic one which is the domain of formative assessment. While formative 
assessment can be used to take corrective measures and monitor progress when learning is still 
happening, summative assessment is used to determine effectiveness of a program, students’ 
achievements etc. after learning has been completed. For that purpose summative assessment 
places more emphasis on accountability with the assignment of a grade. In other words it 
“summarizes ... learning for the purpose of accountability, taking a snapshot in time of their 
performance” (Earle, 2014, p.218). This explains why it is also referred as assessment of learning 
as opposed to formative assessment which is referred to as assessment for learning. 

These two forms of assessment are not mutually-exclusive (Emanuel, Robinson, & Korczak, 
2013). While learning analytics can play major roles in summative and formative assessments, 
research exploring its usage, especially for formative assessment purposes, is limited. Zupanc, 
Urank, & Bren, (2009) described a process where learning analytics is used for both summative 
and formative purposes to “disseminate ... effective tools for assessing the quality of educational 
establishments as feedback on improvement” (p.92). The authors advocated the combination of 
summative and formative assessments through a balance between assessment for learning and 
assessment of learning. 

Romero-Zaldivar, Pardo, Burgos, & Delgado Kloos, (2012) on the other hand seem to focus more 
on the formative assessment aspect and define learning analytics as the use of “data and any other 
additional observations that can be obtained, and use it to directly impact the students, the 
instructors and the details of the learning process” (p. 1059). Referring to previous work from 
scholars (Maki, 2002; Banta, Jones & Black, 2009; Kirkpatrick & Kirkpatrick, 2013), the authors 
suggested the following process shown in Figure 1. Figure 1 depicts the process of collecting and 
analyzing formative and summative assessment data. 
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Figure 1. Steps in Formative and Summative Assessment 

Assessments and Activities in Online Learning 

The development of online learning in higher education requires schools and teachers to shift their 
thinking and practices in terms of learning effectiveness (Gikandi, Morrow, & Davis, 2011; 

Prineas & Cini, 2011). The asynchronously online learning courses provide opportunities for a 
student centered approach to learning and assessment. In fact, the online learning environment 
provides a platform for more performance based assessment through immediate feedback, 
opportunities for individual practice and guidance (Reeves, 2000). These opportunities provided 
by the online learning environment place educators in the ideal and most desired position of not 
only monitoring learning events as they happen but also the possibility to take corrective measures 
and adjust teaching to improve student learning (Romero-Zaldivar, Pardo, Burgos, & Delgado 
Kloos, 2012). Other benefits that the online learning platform provides with respect to assessment 
include better monitoring opportunities for student learning and immediate feedback (Buckingham 
& Ferguson, 2012 ; Benson, 2003 ; Romero-Zaldivar, Pardo, Burgos, & Delgado Kloos, 2012), 
and individual practice opportunities (Shuey 2002 ; Orme, 2004). Scholars (Blummer, 2007; 

Orme, 2004)) have studied assessment practices and techniques that could foster learning 
effectiveness and continuous improvement in an online learning environment. 
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Four types of online learning assessment techniques are commonly used. They include: 
Comprehensive-type, discussion board, reflective-focused, and project-based assessments. These 
assessments can be used both as a formative assessment (activity) or summative assessment. 

Comprehension-type assessment: These are selected response type of assessment. In this type of 
assessment, students are usually given choices to select one or more answers from many. 
Comprehension type assessments aim for students to understand remember and/or memorize 
concepts and ideas (Van den Broek et al., 2005). They are mostly meant for formative type 
assessment as they are quicker tools to gather information that can be used to monitor student 
learning and make necessary adjustments (Cornelius, 2013; Wormeli, 2007 ; Tomlinson, 1999). 
These include assessment such as multiple-choice, true false, matching ranging etc. 

Discussion board: Discussion boards are well-suited for promoting collaboration and interaction 
among online learners. According to Shuey, (2002), these can be used to assess skills such as 
reasoning, collaboration, negotiation, argumentation, and debating (dark, Sampson, Weinberger, 

& Erkens, 2007) and teamwork etc. This method of assessment promotes active learning and also 
allows student to support each other in the form of a learning community and therefore assists 
developing multiple perspectives (Gikandi et al., 2011; Mackey & Evans, 2014) 

Reflection-focused assessments: These assessments focus not only on the correct answers of a 
given problem, but they emphasize the thought processes that lead to that answer (Frederick, 

2002). These are assessments for which students are expected to articulate more elaborate 
responses to questions related to skills and knowledge learned in the course. These assessments 
allow students to formulate their responses using theoretical and practical knowledge. Examples of 
these assessments include short answers, essays, minute papers, research papers, reflection papers 
etc. Cumulative assessments such as eportfolios can also promote students reflective skills in 
addition to helping them connect different learning events and opportunities that happen during a 
course. 

Project-based assessment: Project-based learning (PBL) organizes learning around a project and 
involves answering authentic, real-life challenging questions involving students in constructive 
investigation (Thomas, 2000). Assessing project-based learning is referred to as project-based 
assessment. These assessments include presentations and products. These types of products 
require manual grading based on rubrics. Examples of such skills include the ones requiring 
students to integrate different skills to create a product (a business plan, or lesson plan) or related 
to oral communication (i.e interacting with audience, keeping eye contact, tone of voice etc...) 
which cannot be assessed from written work since they require live or recorded demonstration. 
Projects based assessment provides an opportunity for students to work in groups or individually, 
and the interaction among group members can be analyzed as part of this assessment. Examples of 
project-based assessment in this sample course included, using livebinder to compile a list of 20 
websites that they can use in their future classroom, using smore to create a flyer on computer 
security for the students and parents etc. 

Different type of Assessments and type of Learning Analytics techniques. 

The different types of assessment (comprehensive type assessment, discussion board, reflection 
focused and project based assessment) and the learning analytics techniques and data measures are 
depicted in table 1. These are some techniques that can be used but this is not an exhaustive list of 
all the techniques. Due to time and space constraints we provide below some example techniques. 
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Table 1. Different types of Assessments and Learning Analytics 


Types of Assessment 

Learning Analytics Techniques 

Data Measures 

Comprehension type 

Quantitative Analysis 

• 

Current Score 

assessment 

• Descriptive Statistics 

• 

Time Spent 


• Item analysis 

• 

Frequency of access 

Discussion board 

Social Network Analysis 

Interaction measures 


• Interactions between 

• 

Frequency of Posts 


student and facilitator 

• 

Length of Posts 


• Interaction among 

• 

Themes 


students 




Qualitative Analysis 

Quality 

of Posts 


• Discourse Analysis 

• 

Use of concept and 


• Conversation Analysis 


theories 



• 

Common patterns 



• 

Repeating events 



• 

Key phrases 

Reflection focused 

Qualitative Analysis 

Quality 

of Reflection 

assessments 

• Content Analysis, 

• 

Rationale 


• Concept Mapping 

• 

Multiple Perspective 


• Document Analysis 

• 

Supporting theories or 




frameworks 



• 

Common patterns 



• 

Repeating events 



• 

• 

Key phrases 



Writing Skills 



• 

Grammatical Errors 



• 

Typos 



• 

Coherence of Ideas 



Quality 

of Evidence 



• 

Analysis of Artifacts 



• 

Type of Artifacts 



• 

Justification of Artifacts 

Project based 

Quantitative Analysis 

• 

Current Score 

assessment 

Observation 

• 

Time Spent 



• 

Frequency of access 


Providing Feedback in Assessments and Learning Analytics 

Referring to SOLAR definition provided on the first page of this article, learning analytics seems 
to place a great deal of importance on feedback for teaching and learning effectiveness. The 
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purposes of "understanding and optimizing learning and the environment in which it occurs", as 
reflected in that definition, will depend mostly on providing information in the form of feedback 
to teachers about their teaching effectiveness and to students regarding their learning 
achievements. Providing feedback is very critical in any assessment and feedback provided should 
influence the quality of student work. ‘ 

Feedback is defined as “information about the gap between the actual level and the reference level 
of a system parameter which is used to alter the gap in some way” (Ramaprasad 1983, p. 4). 
Feedback has been studied for decades and several principles on feedback have been identified to 
facilitate learning. Nicole and Macfarlane-Dick (2004) list the following as some of the principles 
of feedback. 

“Feedback is used to encourage teacher and peer dialogue around learning; help clarify 
good performance; provide opportunities to close the gap between current and desired 
performance; deliver high quality information to students about their learning; and also 
provide information that teachers can use to help shape their teaching.” (p.3) 

This paper supports the feedback model proposed by Boud and Molloy (2013) where teachers are 
the drivers of feedback. In online courses, there is a lot of rich data that is being captured by the 
learning management system. Several researchers (Price, Handley, Millar, & O'Donovan, 2010; 
Carless, Salter, Yang, & Lam, 2011) have developed interventions with new assessment activities 
to provide feedback. Learning analytics techniques encourages the use of data to make decisions 
that will assist student learning by bridging any gap that exists between the actual level and the 
reference level. 

Purpose of this Study 

The purpose of this study was to identify learning analytics techniques and data measures for 
different assessment types in online courses. Using a set of structured and non-structured data the 
study suggests different techniques and analysis that can be used to provide feedback that could 
enhance both online teaching and learning. 


Method 


Data Measures 

Sample assessment data measures (as referenced in Table 1) were collected from a preservice 
instructional technology course taught at a southeastern university in the United States. The course 
was taught in a 15 week time period and had 7 modules. Each module included a variety of 
instructional components including an elearning module, a quiz, and hands on projects. There 
were 18 students in this online course and Table 2 below provides the names of the different 
modules in this course and the different assessments that were used. 

Table 2. Modules in the Instructional Technology Course 


Module 

Module Name 

Assessment 

1 

Technology Integration 

20 websites on LiveBinder 
Quiz 1 

2 

Computer Networks, Security and Ethics 

Brochure on Smore 

Quiz 2 

3 

Productivity Applications 

Multimedia Program using 
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PowerPoint 

Quiz 3 

4 

Hardware for Educators 

Budget on Word 

Quiz 4 

5 

Curriculum Integration 

Smartboard lesson using 
Smartnotebook 

Quiz 5 

7 

Online Teaching 

Online Lesson on Edmodo 
Quiz 6 

8 

Eportfolio Technologies 

Eportfolio on Weebly 

Quiz 7 


The types of assessment provided in Table 1 could generate quantitative or qualitative data 
depending on their nature and set up. In this section, different techniques will be presented for 
both types of data. 

Tableau© for Quantitative Data Analysis and Visualization 

The data was imported into a data visualization tool. Tableau©. Different visualization techniques 
were applied. Below is a screenshot of the tableau© software. The analysis techniques shown in 
this article aim to provide instructors with an opportunity to connect different pieces of 
information that can support and inform their decision making in their efforts to formatively 
evaluate teaching and learning and provide feedback to students for more effective teaching and 
learning. Figure 2 is a screenshot of the data analysis tool tableau© which was used to conduct 
quantitative learning analytics. 
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Figure 2. The Tableau© Screen 

ManyEyes© for Qualitative Data Analysis Visualization 


Qualitative data were imported into IBM’s ManyEyes© to run different visualization techniques. 
Below is a screenshot (Figure 3) of the Manyeyes Visualization tool. 



manyeyes VISUALIZATIONS DATASETS Sign In 


Create a Visualization 

It's easy: upload some data, pick a visualization and share your insights with the worldl 



Step 1: Add your data. 

Upload tha data you want to visualize 


i 


Figure 3. Screenshot of Many Eyes© 


Results 


Comprehension-Type Assessment 

Two different example analyses are presented for comprehension type assessment. Time spent 1 
on quiz 1, quizl score and frequency (number of times) of access were tabulated and a whisker 
plot was drawn (see Figure 4). The maximum score that a student could earn in quiz 1 was 10 
points. The quiz included 10 multiple choice items with four item responses. Since this course was 
offered 100% online, the students had the option to take this quiz open book. There was no time 
limit set for them to complete the quiz. The whisker plot provided representation of variables by 
providing the median values for each one of them. In this visual below, the median time spent on 
the quiz is between 0.03 and 0.35 (hours), and the median quiz 1 score is between 8 and 9 points, 
and the median for the number of times accessed was 4. Analysis like this can provide instructors 
with useful information on students’ behavior. For example if you look at the element “number of 
times accessed” the figure shows that he lower quartile is 3 times and the upper quartile is 6 times 
meaning half of the students (50%) accessed the quiz between 3 to 6 times. The average score is 
8.44 with the minimum score being 6 and the maximum score being 10. Such information could 
be useful and meaningful for instructors, as it could be used to benchmark students ‘scores and 
practices (i.e. time spent on a quiz) to analyze and understand students’ performance in a course. 

In each of the three areas (time spent on quiz. Quiz 1 score, and number of times accessed), the 
dots outside the box represent outliers. 
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00 

Time Spent on Quiz 

O 

in o 

w 

1 

• MB 


Quiz 1 score 


Number of times accessed 


tn 



tn 






Figure 4. Box and Whisker Plot depicting time spent on Quiz 1, Quiz 1 score and Frequency 
(number of times) of Access of Quiz 1 

Following this preliminary analysis, we looked at the possible relationship between the number of 
times the quiz was accessed and the quiz score (see Figure 5). The goal of this analysis is to 
identify the type of information that might be useful to an instructor. For example the visual shows 
that, the student who scored 6 points on the quiz, accessed it four times, and another student who 
scored the highest accessed it between two or three times. From examining the visual, an 
instructor might be able to relate pieces of information such as how many times a student accessed 
an assignment with their final score on that assignment in an attempt to understand factors that 
might influence a student's performance. This scatter plot can benefit an instructor as it can be 
used to relate or explain student performance with respect to all these behaviors, therefore help 
provide students with targeted feedback they can use to improve their learning performance. 
Different variables can be entered in the rows and columns of tableau© and different 
representations can be drawn. For example, figure 5 tells us how many times a student accessed 
the quiz but does not tell us how long they stayed. 
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III Columns 
i= Rows 


SUM (Quiz 1 score) 
SUM(Number of times 



student14 

i 


? 

$tudent7 


S 

8 6 
s 6 
s 

E 

i 

S 5 


o 

student8 


0» student 13 


* student2 


V 


<9» studentl 


9 

studentl 1 


Y 

student4 


studentl 7 


5 6 

Quiz 1 score 


Figure 5. Scatter Plot depicting Quiz 1 score and number of times accessed 

Further analysis was conducted by relating quiz 1 score and time spent (figure 6 below). With 
such analysis an instructor could follow individual students to detect patterns. For example 
students 5 and 6 who got one of the lowest scores on figure 5, accessed the quiz 4 times and 5 
times respectively but did both only spent less than 0.2 hours, and their scores of respectively 6 
and 7 are both below the median quiz 1 score which is between 8 and 9 points reported earlier in 
figure 4, and also below the average score of 8.44. 
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iii Colur 

nns SUMCQuiz 1 score) 


Rows 

SUM (Time Spent on Quiz) 


1.4 

1.3 


student4 

studentll 

student3 

<5 

1.2 


^_^ student14 

1.1 

1.0 



0.9 

.a 0.8 
o 

S 

£ 0.7 

; i 

*— 0.6 

0 5 

0.4 


student7 q 

0.3 


studentIB ., student2 

0.2 


G*> studentl 3 

student8 studentl 

0.1 


. _ ^ student9 

students studentl6*- 

° . 0_ « 

^- - — studentl 7 

0.0 


students studentIO student12 studentl5 


0123456789 10 

Quiz 1 score + 


Figure 6. Scatter Plot depicting Quiz 1 score and time spent 



Figure 7. Bullet graph depicting frequency of access of Module 1. Higher the frequency of access, 

bigger the bullet size 
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This could indicate that these students though accessing the quiz are not spending enough time on 
it. Therefore feedback for these students could be modeled around taking enough time to complete 
the quizzes. Student 14 who also has one of the lowest scores (7) which is below the average score 
of 8.44 and accessed the quiz 10 times (figure 5), spent more than 1 hour. This also could indicate 
that this student though coming in frequently did not spend enough time every time she or he 
accessed the quiz. Student 4 on the other hand who has one of the highest scores accessed the quiz 
about 3 times and spent a little bit more than 1.3 hours. Students 17, 15 and 9 on the other hand 
show a different pattern as they accessed the quiz fewer times (no more than twice), spent no more 
than 0.2 hours and ended up with some of the highest scores. An instructor could use all of this 
information to develop more targeted support, feedback and instruction if needed. For example 
feedback for student 6 might be focusing on spent ding more time and figuring out the reasons 
why she or he did not spend enough time. Information collected through this feedback process and 
dialogue could be used to adapt instructions and teaching for these students if needed. 

Project-Based Assessment 

Time spent on project 4 and Project 4 score was tabulated and side by side bar analysis was 
performed. The grades that the students earned were distributed between 11 and 15. There was one 
student who received no points. The average score was 13.16 with the minimum score being 11 if 
we exclude the student with a 0 and the maximum score being 15. With the information such as 
the one represented in Figure 6, an instructor can analyze students' score with respect to how much 
time they spent on this assignment to see if there might be any emerging patterns, as well as 
feedback for him or herself and the students. Moreover an instructor could use this information to 
answer questions such as whether time spent affect a student's score or what seems to be the 
optimal time spent for a student to achieve a high score? 

A bullet graph (see Figure 7) is another way to provide an instructor with a quick glance at the 
frequency of access to show who accessed it the most and who accessed it the least based on the 
size of the bullet. From the above visual, student 18 had accessed module 1 the most number of 
times, whereas student 5 and 9 had accessed it the least number of times. This information could 
be useful for an instructor who would like to detect any patterns of continued and sustained efforts 
in learning. Information provided in Figure 7 could be related to other students' behavior or 
performance. For example an instructor could use this information not only to give students’ 
feedback on their performance but also on their efforts and participation. Additionally similar 
analysis as the ones conducted for quiz 1 was also performed. 
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Figure 8. Scatter Plot depicting Project 4 score and number of times accessed 

Using information reported on figure 9, an instructor could see results indicating that student 6 is 
again the one with the lowest score and also among the ones who access the project 4 the least (2 
times). Further that same student 6 only spent 2.5 hours on the project (see figure 10 below). 
Student 4 accessed the project 3 times and spent 3.5 hours and consequently earned one of the 
highest scores. 

Student 14, who had one of the highest accesses on the quiz and the lowest scores as well, shows a 
different pattern for the project which he or she accessed only 3 times (figure 9) and got a score of 
13 out 15 slightly below the average of 13.16. Comparing the results on figure 9 and figure 10, it 
appears that most of the students who got the highest scores (students 11, 18, 17, 4 and 15) spent 
at least 3.5 hours on the project which they accessed at least twice. Students 16 and 13, on the 
other hand spent less than 1 hour even though they earned some of the highest scores. A quick 
glance at figure 9 also revealed that those are among the students with the highest frequency of 
access of module 1 which was the content related to the project. Moreover a comparison between 
figures 5 and 8 show that students 7 and 18 shows a pattern of high frequency of access for both 
the quiz and project. Student 15 on the other hand seems to show a pattern in low frequency of 
access for both the quiz and the project. 
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SUM (Project 4 Score) 
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„ 4 0 
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s. 
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studentl? 
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student14 


studentIO . 


X 
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student4 
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X 
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studentl 


student 16 


\ 


studentl 3*0 

14 IS 16 


Project 4 Score 


Figure 9. Scatter Plot depicting Project 4 score and time spent 

Reflection-Focused Assessment 

Assessment for reflection-based assignments can focus students’ use of key phrases as indicated in 
Table 1. In this case students’ reflections were collected from their e-portfolio and entered in 
Many Eyes©. Figure 10 below shows the main key words and concepts most used by students. 

The bigger the font size of the words, the more frequent its use by students. It shows the words 
students, children, educator and standards were the most represented ones in the reflections. This 
analysis could be used by instructors especially if they want to see students' usage of certain key 
words and concepts. For example in this case, the higher frequency of words such as children, 
educator and standards, could be due to the fact that students in this case were teacher candidates, 
as these are very important concepts in the teaching profession. The same analysis could have 
revealed other words or concepts in different fields. 

Results can be sorted in different ways, such as order of appearance, by frequency, alphabetically 
etc. Results displayed are also interactive and it is possible to eliminate some of the least 
represented keywords from the analysis. Similar analyses can be conducted using other software 
such as Excel Textalics which can perform topics extraction to see which topics are the most 
common, text classification etc. 
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Figure 10. Word Cloud visualization of key words used by students in their eportfolio reflections. 

Discussion Board Assessment 


Similar analysis can be conducted with discussion topics. Analysis can reveal most common 
words, types and categories of words used. For the introductory discussion, students were asked to 
introduce themselves using the discussion board. Along with introducing themselves, they were 
also asked to share what they hope to learn from the course, and share something or someone 
special to them. In the example below, students introductory statements from an online class were 
entered into many eyes©. From the Figure 12 below, it appears that students in this class refer to 
family or other words related to family such as sisters a lot in their introductions to other 
classmates in response to something or someone special. This type of information can be very 
useful for instructor to have a better understanding of the types of students they have as well as 
background information and other factors that may affect learning in the classroom. For example 
results from the Figure 11 below might suggest that students in this class place a lot of emphasis 
on family. 
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Figure 11. Chord Chart Visualization of Introduction discussion posts 

Discussion 

This article provides guidelines and suggestions for the necessary shifts in how instructors, 
especially those involved in online learning, need to approach student learning using all available 
and meaningful data at their disposal. The main characteristics of such shifts are to promote 
simultaneous and instant use of data for feedback to students as well as effective decision making 
during learning as opposed to using such data only after the learning event. With learning analytics 
techniques, such as the ones displayed here, the instructor no longer need to wait for the end of the 
course to download and analyze the data. The instructor also does not have to wait for another 
employee to run these analyses. The instructor can download the data, analyze it, draw 
conclusions, and act on it immediately as learning is happening. This possibility to act on 
instantaneous data places the instructor at an ideal position in the quest to improve student 
learning (Arnold & Pistilli, 2012; Becker, 2013). Further this type of information can help online 
instructors predict and identify at risk students and therefore develop strategies to remedy the 
situation. 

Data displayed in the figures above can help instructors gather longitudinal data in order to 
observe trends and follow each student, especially in small groups. For example follow student 6 
online behaviors and practices could be traced through the quiz and the project to detect patterns. 

Based on the examples, analyses and tools shown above, learning analytics could have multiple 
benefits for online course instructors as well as students. Online course instructors can use 
multiple assessment techniques through the lens of learning analytics to support student learning 
through data driven decision making model. According to Dietz-Uhler and Hurn (2013), learning 
analytics can be used to monitor student behaviors and promote their success in online learning. 
This is very important in online learning format given the fact that the instructor might not have 
the time to meet with students as regularly as one would in a face to face course. Therefore the 
instructor would need to have access to meaningful information that would allow quick reaction. 
For example from the figures above, an instructor could have an array of information available to 
see if there are any patterns or useful information such as how much time students spent or how 
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many times they access the quiz and their final score. Information of this type can help an 
instructor to find trends and decide what types of actions need to be taken to address shortcomings 
or build on potential benefits. For example, using the information provided in figure5 and 9, an 
instructor could detect pattern that student 6 not only had the lowest score on quiz 1 but is one of 
the student also who accessed the quiz the fewest. With such information, an instructor could 
reach out to that student via email, phone call, or even a skype meeting to help identify barriers 
and issues preventing access as well as expected performance. Instructors could even suggest a 
face to face meeting with such a student if it is convenient for both of them, regardless of the 
course being online. However, with such a small sample in this case study inferential claims 
cannot be made. If teaching large class sizes and if vast volumes of data are available these types 
of inferences can be made. 

Similarly instructors of online courses can gather information from students’ introductions to see 
what type of students are enrolled in the class, what things do they seem to value, and what factors 
might interfere with their learning. All this information once gathered from analysis like the Chord 
Chart in Figure 10, can be used by online instructors to better understand students’ background 
that otherwise would have been difficult to have access to without a face to face meeting with 
students. Using this type of information online course instructors can help students develop more 
personalized learning based on their background and circumstances to improve learning (Shum & 
Ferguson, 2012; Zupanc et ah, 2009). Gathering and acting on information obtained from students 
‘interactions can be related to what Shum and Ferguson, (2012) refer to as social learning 
analytics. According to the authors social learning analytics could be understood as type of 
analytics “that draws on the substantial body of work demonstrating that new skills and ideas are 
not solely individual achievements, but are developed, carried forward, and passed on through 
interaction and collaboration’ (p.5). 

The instant and immediate feedback that could be derived from learning analytics techniques and 
tools such as the ones shown above could be very beneficial to students and instructors. One of the 
major benefits is that these learning analytics provide students with the availability of immediate 
and instant feedback they can use while still working on their assignments or projects. In that 
sense instructors will become more proactive in terms of identifying factors that can influence 
learning and acting on them for the benefit of the students. 

While learning analytics in general can be very helpful to an instructor in supporting student 
learning, practitioners need to be aware that it does have some limitations in terms of observing or 
monitoring what happens outside the learning management system. This might be the reason why 
there seems to be no clear patterns at times between scores and time spent or number of times 
accessed. For example time students spend reading textbooks, or other materials to help them 
complete a project are is an important aspect of motivation and a factor that contributes to learning 
but yet cannot be captured with learning analytics in its current form. Similarly students’ abilities 
to engage in reflective activities, to regulate their own learning through goal setting, to monitor 
and evaluate the quality of their work are also important factors of learning that are not captured in 
learning analytics (Winne, 2010). Students This It is to address these limitations that scholars 
(Winne & Peery, 2000) suggested to consider learning as an event and that processes that 
happened during such event can assessed through students’ self-report and recounting of them. 
This is one of the reasons, the researchers suggested individual one on one meeting with student 6 
as one of the ways to account for such factors. 

Implications and Future Directions 
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The goal of this article is to report the results of a small case study of 18 students in an online 
environment. Some of the data available from the 18 students’ use of the online learning 
platform are exported and quantitative and qualitative visualizations of learning analytics data are 
presented. Furthermore, the information and feedback provided through the use of learning 
Analytics data for assessment in online and blended learning could be of great importance to all 
stakeholders. 

As an instructor, taking the time to review the learning analytics data on student activity and 
assessment was meaningful though the class size was small. It helped identify the students who 
were very active in the online class and were spending a lot more time in the Learning 
Management System. It also helped identify students who were not as active and who I needed to 
reach out to both in terms of performance and engagement. Analyzing the data helped with the 
implementation of similar courses and in designing the assessments in a way that is more 
beneficial to the students. As an instructor, if I noticed that there was a student who was 
struggling with a particular module, I reached out to the student to provide additional support. I 
was also able to reach out to inactive students early on, rather than waiting until the end of the 
semester to provide support and develop remediation strategies. 

While instructors may use such information for effective online teaching, students can also use it 
to enhance their learning. Instructional designers may use this information to recommend best 
practices in online course design. Administrators may use this information to design successful 
online programs. Educational researchers may use this framework to analyze data from the 
various online assessments within the learning analytics framework. Bringing together these 
points of view will help improve online teaching and learning. 

Future directions for this study will be to conduct research on the effectiveness of these tools. 

The authors also plan to implement the various data analyses techniques in large enrollment 
classes in the future. 
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